Autoencoder for wind power prediction

Successful integration of renewable energy sources like wind power into smart grids largely depends on accurate prediction of power from these intermittent sources. Production of wind power cannot be controlled as the wind speed can vary based on weather conditions. Accurate prediction of wind power can assist smart grid that intelligently decides on the usage of alternative power sources based on demand forecast. Time series wind speed data are normally used for wind power prediction. In this paper, we have investigated the usage of a set of secondary features obtained using deep learning for wind power prediction. Deep learning is a special form on neural network that is capable of capturing the structural properties of time series data in terms of a set of numeric features. More precisely, we have designed a two-stage autoencoder (a particular type of deep learning) and incorporated the structural features into a prediction framework. Using the structural features, we have achieved as high as 12.63% better prediction accuracy than traditionally used statistical features. © The Author(s) 2017. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Introduction Renewable energy sources like wind are becoming integral part of modern power systems. As reported in IRENA (2017), renewable energy accounts for around 22% of global power generation. This share is expected to double in the next 15 years. This is due to the rapid growth of variable renewable energy from sources like wind and solar photovoltaic (IRENA 2017). Renewable energy offers several advantages such as easy availability, applicability, and environmental friendly. The application of smart grid in renewable energy makes it even more promising. Smart grid engineering is the key for a beneficial use of widespread energy resources. This fusion of smart grid and renewable energy enables the efficient use of such sources. Alongside offering the opportunities, integration of renewable energy like wind power into smart grids is not without challenges. The key issue is being the intermittent nature of wind power. The wind speed varies and so is the power produced from wind-driven power station. Also the produced energy needs to be consumed immediately unless that is stored at additional cost. It is thus highly beneficial to know in advance the amount of wind power that can be expected. It is also important from demand management point of view. Fossil fuel supplies for power generation can be adjusted based on expected demand. That is, however, not possible for wind energy sources for the reasons explained above. Prediction/forecasting of wind power is thus a necessity for integrating wind energy into smart grids. Wind power prediction methods are developed to deal with this problem and aim to predict generated power based on historical weather/wind data by utilising data mining methods (Wang et al. 2011, 2016; Colak et al. 2012; Soman et al. 2010; Zhao et al. 2016; Jiang et al. 2017). In general, historical wind data obtained from weather stations are used by data mining algorithms to make the predictions. Wind data over time is time series data. Traditional data mining approaches model predicted wind power as a function of raw wind data over a period of time. A wind power prediction method was previously attempted in Tasnim et al. (2014) by modelling predictions as a function of statistical features extracted from raw time series data. Promising results were reported when the ensemble feature-based prediction framework was adopted. The trend of investigating new feature representations for day-ahead wind power prediction is continued Open Access *Correspondence: [email protected] 2 Data61, CSIRO, 15 College Rd, Sandy Bay, TAS 7005, Australia Full list of author information is available at the end of the article Page 2 of 11 Tasnim et al. Renewables (2017) 4:6 in our research presented in this paper. In this particular research work, feature representations are learnt using a particular kind of deep learning algorithm called stacked autoencoders (Ng et al. 2016; Bengio et al. 2007; Shin et al. 2014). Autoencoders generate a compressed lowdimensional structural representation of the time series (Bengio et al. 2007). A stacked autoencoder obtains structural representations (i.e. features) at multiple stages by repeated application of autoencoders on the compressed feature space. Supervised learning algorithms are trained on the compressed feature space. State-of-the-art learning performance was achieved by stacked autoencoders on images (Vincent et al. 2010; Gehrig et al. 2013), speech (Gehrig et al. 2013), agricultural applications (Rahman et al. 2016), and other structured time series (Shin et al. 2011) signals. This paper investigates whether the stacked autoencoder provides an effective representation for wind power prediction. In previous studies (Tasnim et al. 2014), an ensemble framework was considered for wind power prediction. For the sake of completeness, we also embedded the autoencoder features in cluster-based ensemble framework in Rahman and Verma (2011) and Rahman et al. (2010) and investigated its effectiveness as part of the framework too. To the best of our knowledge, incorporation of autoencoder features in day-ahead wind power prediction framework is a novel idea and we consider this as the key contribution in this research. We have investigated the following research questions in this paper: (1) investigating the effectiveness of autoencoder features for wind power prediction, (2) comparing the performance of autoencoder features to statistical features for wind power prediction, and (3) how much improvement do we achieve by embedding the autoencoder features in ensemble framework. Experimental results reveal that we achieved as high as 12.63% improvement by using autoencoder features over statistical features. Proposed prediction framework The prediction framework has normally two components: training and prediction module. During training, historical time series data are split into small time windows and prediction targets are set for each window. Feature vectors are computed from each time window. This produces a 2D (two-dimensional) matrix where each row represents a feature vector. The targets are presented in a column vector where ith entry is the target for the ith row in the 2D matrix. A regression algorithm is trained on these matrices to produce a model that can reproduce the targets (with minimum error) given the input vectors from the 2D matrix. During prediction, data available up-to-date are windowed and presented to the regression model to produce the predictions in the future. In this paper, we have investigated autoencoder features and also their effectiveness as part of cluster-based ensemble learning algorithms. We present both in this section. For the sake of completeness, we present the statistical features as well. Statistical features We need to specify the structure of the input vector and target for training the regression models. We have used wind power as the target that needs to be predicted. Let ws = (ws0, ws1, ..., wsn−1) be the vector representing the wind speed over n consecutive days. A set of m statistical features s = s1, s2, ..., sm are computed from the wind speed vector ws and the vector s as the input vector for the regression algorithm. The features were computed from the time and frequency domain representations of the wind speed vector ws. Discrete Fourier transformation (DFT) was applied on ws to obtain the frequency domain representation of the time series data. Let wst be the tth element of the time series. The jth element of the frequency domain representation is obtained as: where n is the length of the vector. Here ws represents the wind speed at various points in time and f represents the signal strength at various frequencies. We have used the DC (direct current) component of the DFT ( f0: component corresponding to 0 frequency) as a feature. A set of statistical features are then computed from the remaining high-frequency (> 0) spectrum of f . The following statistical features are computed: mean, standard deviation, skewness, and kurtosis. We also used minimum and maximum of the series ws and f as features. The standard deviation, minimum and maximum features were used to represent the intensity. A total of 13 statistical features were computed from ws and f . Autoencoder features An autoencoder (AE) is one form of deep learning algorithm (Ng et al. 2016; Bengio et al. 2007; Shin et al. 2014). AE can be considered as an unsupervised variant of a neural network with one hidden layer where the target vector is set to be equal to the input vector. AE thus tries to learn an identity function. However, by reducing the number of nodes (compared to input) in the hidden layer, interesting structural features can be learned (Bengio et al. 2007). Normally a backpropagation algorithm is applied to learn the weights in the network. For the wind power prediction problem, the AE will try to learn a function Iθ ,b such that Iθ ,b(ws) ≈ ws where ws is the wind speed vector, θ and b are the (1) fj = n−1 ∑